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ABSTRACT 

This  paper  presents  the  new  developments  of  the  next 
generation  of  the  digital  human  Santos.  The  new  features 
include  (1)  a  211-degree-of-freedom  realistic  skeletal 
model  with  deformable  skin;  (2)  advances  in  a  new 
method  for  dynamic  motion  prediction,  called  predictive 
dynamics ;  (3)  advances  in  strength  and  fatigue  modeling; 
and  (4)  advances  in  virtual  human  clothing  interaction 
simulation.  With  these  new  developments,  Santos  can 
simulate  posture  and  motion  with  higher  accuracy,  predict 
realistic  cloth  interaction,  consider  strength  and  fatigue 
factors  in  predictive  dynamics,  and  facilitate  improved 
and  more  efficient  product  design.  In  addition  to 
providing  new  developments  with  various  aspects  of 
human  modeling,  this  paper  also  highlights  two  high-level 
approaches  to  human  modeling.  First,  predictive  human 
modeling  is  addressed  on  the  joint  level,  as  opposed  to  the 
muscle  level.  This  is  especially  novel  with  respect  to 
strength  and  fatigue.  Secondly,  all  aspects  of  human 
modeling  either  affect  or  are  affected  by  motion 
prediction;  predicting  motion  on  the  joint  level  is  the  core 
of  the  comprehensive  human  model.  Ultimately,  Santos 
can  be  deployed  in  different  fields  to  serve  various 
customers. 

KEY  WORDS 

Digital  human;  predictive  dynamics;  muscle  strength; 
muscle  fatigue;  clothing  modeling;  skeletal  model 

1.  Introduction 

Digital  human  modeling  is  becoming  increasingly 
important  to  today’s  designers  and  manufacturers. 
Determining  human  performance  in  terms  of  a  workplace 
or  a  product  before  it  exists  ensures  consistence  with 
health  and  safety  standards,  accelerates  time-to-market, 
increases  productivity,  and  reduces  design  timeframe  and 
associated  costs. 

Commercial  human  models  available  in  the  market  are 
Jack®,  Ramsis®,  and  Safe  work®.  Jack  enables  users  to 
position  biomechanically  accurate  digital  humans  of 
various  sizes  in  virtual  environments,  assign  them  tasks, 
and  analyze  their  performance.  Ramsis  specifically  targets 
the  automobile  industries.  The  core  capabilities  of  this 


software  are  the  realistic  display  of  international 
anthropometric  data  and  the  efficient  analysis  of 
ergonomic  questions  concerning  sight,  maximum  force, 
reachability,  and  comfort.  Safe  work  structures  multiple 
human  modeling  systems  to  facilitate  detailed 
investigation  into  human-centered  design  issues.  Other 
research  and  development  digital  human  models  include 
SAMMIE  developed  by  Porter  et  al.  (1999),  the  Boeing 
Human  Modeling  System  (Rice,  2004),  and  Dhaiba 
(Mochimaru  et  al.,  2006).  However,  all  of  these  human 
models  are  based  on  experimental  data. 

To  overcome  the  deficiencies  of  the  current  models,  we 
are  developing  a  new  generation  of  digital  human,  Santos. 
This  model  is  based  on  optimization  techniques  instead  of 
prerecorded  data.  Consequently,  it  provides  a  tool  for 
studying  how  and  why  humans  move  as  they  do.  However, 
the  human-body  system  is  truly  multi-scale,  and  any 
effort  to  model  humans  and  their  limitations  must  be 
multi-disciplinary.  Consequently,  Santos  involves  a 
variety  of  research  and  development  projects.  These 
include  (1)  skeletal  modeling;  (2)  predictive  dynamics;  (3) 
clothing  modeling;  (4)  strength  and  fatigue  modeling;  (5) 
physiology  modeling;  (6)  an  advanced  hand  modeling 
package;  (7)  posture  prediction  with  multi  end-effectors 
and  real-time  inverse  kinematics  (IK);  and  (8)  muscle 
wrapping  and  muscle-force  determination.  In  our  previous 
reports  (Abdel-Malek  et  al.,  2006a;  Abdel-Malek  et  al., 
2006b;  Kim  et  al.,  2006;  Man  et  al.,  2006  Marler,  2005; 
Patrick,  2005;  Yang  et  al.,  2005;  Yang  et  al.,  2006a;  Yang 
et  al.,  2006b)  we  demonstrated  these  capabilities.  In  this 
paper,  we  report  on  new  developments  with  the  skeletal 
model,  strength  and  fatigue  modeling,  human-clothing 
interaction  modeling,  and  predictive  dynamics.  This  paper 
also  discusses  how  the  various  components  of  human 
modeling  are  coupled  together.  One  of  the  key  elements 
of  enabling  this  coupling  is  viewing  predictive 
capabilities  at  the  joint  level  rather  than  muscle  level  or 
spatial  (Cartesian  points)  level.  This  is  especially  helpful 
with  modeling  strength  and  fatigue,  and  relating  such 
models  to  motion  prediction. 

2.  New  Skinned  Skeletal  Avatar 

This  section  discusses  the  newly  developed  skinned 
skeletal  avatar.  It  consists  of  skeleton  and  skin. 
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2.1  Skeleton 

A  potential  restriction  with  any  whole-body  human  model 
is  the  fidelity  of  the  underlying  musculoskeletal  system. 
Consequently,  a  new  system  has  been  developed  for 
Santos  that  mimics  an  actual  human  skeleton,  providing 
the  highest  possible  fidelity  with  respect  to  the  number  of 
degrees  of  freedom.  The  original  kinematic  skeleton  used 
to  predict  gross  human  posture  and  motion  in  2003  was 
dictated  by  the  preliminary  mathematical  predictive 
models  that  had  already  been  developed.  The 
requirements  of  the  predictive  models  at  that  time 
specified  a  total  of  15  degrees  of  freedom  (DOFs).  While 
not  biomechanically  accurate,  this  kinematic  skeleton  led 
to  important  advancements  in  the  predictive  mathematical 
models  for  gross  human  posture  and  motion.  Over  time, 
additional  DOFs  were  added  to  incorporate  the  left  arm, 
neck,  and  legs,  as  well  as  to  address  visually  unsatisfying 
postures  and  motion  during  extreme  reaches  resulting  in  a 
kinematic  skeleton  with  109  DOFs.  While  still  not 
biomechanically  correct,  this  kinematic  skeleton  has  been 
used  to  produce  highly  realistic  gross  human  motion  and 
posture  for  the  past  two  years. 

Recent  efforts  at  VSR  required  a  more  biomechanically 
accurate  approach.  Specific  areas  of  interest  include  a 
more  realistic  representation  of  wrist  pronation  and 
supination;  a  shoulder  model  that  addresses  coupling  of 
the  clavicle,  scapula,  and  humerus;  and  joints  in  the 
kinematic  spine  that  coincide  with  significant  anatomical 
landmarks.  While  it  is  obvious  why  a  significant  increase 
in  kinematic  skeleton  complexity  would  require  changes 
in  the  predictive  models,  an  understanding  of  how  a 
polygonal  mesh  (or  skin)  for  any  given  avatar  is  prepared 
is  required  to  better  understand  the  cost  of  this  process 

The  process  of  assigning  groups  of  vertices  of  a  polygonal 
mesh  to  a  hierarchical  joint  structure  so  that  the  polygonal 
mesh  behaves  as  if  it  had  the  material  properties  of  human 
skin  is  well  defined  and  is  commonly  referred  to  as 
“skinning”  or  “rigging”  in  the  world  of  3D  computer 
animation.  It  begins  with  ensuring  that  the  topology  of 
the  polygonal  mesh  (or  skin)  is  optimized  for  human-like 
movement  by  arranging  the  polygons  in  a  way  that 
reflects  how  the  underlying  musculature  expands  and 
contracts  around  each  of  the  joints.  Once  the  skin  has 
been  optimized,  a  hierarchical  joint  structure  (kinematic 
skeleton)  representing  all  the  joints  must  be  developed 
and  is  shown  in  Fig.  1. 

2.2  Skin 

After  the  kinematic  skeleton  has  been  built,  all  the 
vertices  in  the  polygonal  mesh  must  be  bound  to  the 
appropriate  joints  in  the  kinematic  skeleton.  This  requires 
several  weeks  of  dedicated  time  by  a  highly  skilled  3D 
artist  so  it  is  important  to  avoid  rigging  until  it  is  certain 
that  the  kinematic  skeleton  is  correct  (Fig.  2).  However, 
the  bones  of  a  3D  skeleton  do  not  bend  or  stretch  which 


minimizes  the  rigging  requirements  to  simply  grouping 
entire  bones  to  the  appropriate  nodes  of  the  kinematic 
skeleton.  This  provided  valuable  visual  feedback  as  the 
new  kinematic  skeleton  was  developed  and  tested  without 
the  expense  of  rigging  avatars.  Upon  completion  of  the 
new  kinematic  skeleton,  it  was  clear  that  the  3D  skeleton 
model  would  provide  a  useful  aid  in  ensuring  that  the 
locations  of  the  kinematic  joints  for  each  avatar  were 
anatomically  correct  as  shown  in  Fig.  3. 


Fig.  1  Kinematic  skeleton 


The  benefits  of  the  new  kinematic  skeleton  are;  (1)  the 
locations  of  the  joints  are  based  on  bony  landmarks  of  a 
3D  human  skeleton  derived  from  CT  scan  data;  (2)  every 
skeletal  articulation  is  now  accounted  for,  allowing  for 
maximum  extensibility  in  future  research;  and  (3)  this 
biomechanically  correct  skeleton  now  allows  the 
mathematical  models  for  predicting  posture  and  motion  to 
include  deeper  biomechanical  factors,  such  as  individual 
muscle  forces  and  fatigue.  It  should  be  noted  here  that 
while  this  3D  skeletal  model  is  not  accurate  enough  to 
identify  bone  rugosity,  it  is  accurate  enough  to  identify 
significant  anatomical  landmarks. 


(a)  (b) 

Fig.  2  Skin  rigging:  (a)  Automatic  rigging;  (b)  manual 
rigging 

Anatomy  experts  were  consulted  to  visually  confirm  that 
the  location  of  the  3D  skeleton  within  the  mesh  was 
reasonable  before  rigging,  which  is  shown  in  Fig.  4. 

Given  this  foundation  for  the  human  model,  the  next  step 
is  to  model  how  the  human  moves.  This  is  done  using  a 
new  approach  to  dynamic  motion  prediction,  called 
predictive  dynamics. 
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Fig.  3  A  3D  model  of  a  human  skeleton  used  to  identify 
anatomically  correct  locations  for  the  kinematic  joints 


Fig.  4  New  skinned  kinematic  model 


3.  Advances  in  Predictive  Dynamics 

Predictive  dynamics  (Kim  et  al.,  2005;  Kim  et  al.,  2006) 
is  a  novel  approach  for  predicting  and  simulating  human 
motion.  This  optimization-based  approach  avoids  solving 
typical  differential  algebraic  equations  (or  ordinary 
differential  equations)  in  order  to  create  the  resulting 
simulations  for  highly  redundant  systems.  Detailed  and 
anatomically  correct  joint-based  full-body  human  models 
with  high  degrees  of  freedom  can  thus  be  used  to  create 
more  realistic  simulation  of  tasks  with  relatively  less 
computation.  Various  tasks  like  walking  (Xiang  et  al., 
2007)  and  running  (Chung  et  al.,  2007)  have  been 
simulated  using  this  approach.  In  this  paper,  we  present 
the  general  formulation  and  examples  such  as  stair 
climbing,  throwing,  and  box  lifting. 


3.1  Optimization  formulation 

The  problem  statement  for  each  of  the  dynamic  tasks 
presented  in  this  paper  can  be  stated  as  follows:  “Given 
task-based  parameters,  human  anthropometry,  segment 
inertial  properties,  physical  joint  motion  and  actuation 
limits,  and  desired  time  for  completion,  generate  visually 
appealing  and  dynamically  consistent  task  simulations 
that  minimize  dynamic  effort.”  Such  a  problem  statement 
lends  itself  to  an  optimization  formulation,  various 
components  (design  variables,  performance  measure,  and 
constraints)  of  which  are  discussed  below. 


Performance  Measure 

The  goal  of  the  optimization  process  is  to  reduce  the 
dynamic  effort  at  each  joint.  The  performance  measure 
(or  objective  function)  is,  therefore,  to  minimize  the  sum 
of  the  torque  square  for  all  joints  over  the  simulation  time 
as  follows: 

T  ndof 

/(q)=  j 

t= 0  i=l 

where  ndof  is  the  total  number  of  joints  of  the  human 
model,  t i  is  the  actuator  torque  of  the  zth  joint,  and  T  is  the 
total  simulation  time.  Joint  actuation  torques  are 
calculated  using  a  recursive  Euler-Largrangian 
formulation  as  a  function  of  joint  angles,  velocities,  and 
accelerations. 

Constraints 

Several  physics-based  and  task-based  constraints  have 
been  employed  to  predict  the  motions  for  various  tasks. 
The  physics-based  constraints  that  are  common  for  all 
tasks  are  joint  angle  limits,  torque  limits,  no  ground 
penetration,  dynamic  stability,  and  self-avoidance  that 
ensures  that  different  segments  of  the  digital  human  do 
not  penetrate  each  other  (Xiang  et  al.,  2007). 

3.2  Tasks 

i.  Stair  climbing 

Prediction  and  subsequent  simulation  of  the  stair-climbing 
task  requires  specification  of  additional  parameters  like 
step  length  and  step  height.  Additional  (task-based) 
constraints  imposed  on  the  optimization  problem  are 
described  below: 

•  Symmetry  Conditions'.  One  of  the  assumptions  in 
predicting  stairs  climbing  motion  is  symmetric  and  cyclic 
motion.  Hence,  to  avoid  any  discontinuities  of  the  joint 
angle  profile  for  continuous  motion,  the  initial  and  final 
postures  of  a  step  being  simulated  should  satisfy  the 
symmetry  condition. 

•  Foot  Strike  Position :  Foot  strike  position  is  also  a 
function  of  step  length  and  step  height.  The  distance 
between  the  foot  strike  position  on  the  staircase  and  the 
contacting  points  on  the  foot  should  be  zero  at  contact. 

•  Soft  Impact:  The  impact  created  when  the  foot  lands  on 
the  ground  must  be  minimized  in  order  to  reduce  the  loss 
of  energy.  This  constraint  has  been  imposed  as  zero 
velocity  of  contacting  points. 


Design  Variables 

Joint  angle  profiles,  qt(t)  ,  are  approximated  as  linear 
combinations  of  cubic  B- spline  basis  functions.  Thus,  the 
control  points  representing  the  B -splines  are  the  design 
variables  for  the  optimization  problem.  Corresponding 
joint  angle,  velocity,  and  acceleration  values  are 
calculated  at  each  iteration,  from  these  control  point 
values. 


Fig.  5  shows  sequential  snapshots  of  Santos  climbing 
stairs.  The  motion  appears  visually  realistic.  However,  in 
absence  of  any  collision  detection  strategy,  Santos’  leg 
penetrates  the  stairs  as  seen  in  the  first  and  last  snapshots 
in  the  second  row. 
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Fig.  5  Sequential  snapshots  of  Santos  walking  up  a 
staircase 

ii.  Throwing 

Additional  task-based  parameters  like  mass  of  the  object 
to  be  thrown  and  target  location  must  be  specified  to 
simulate  the  highly  redundant  throwing  problem.  Some  of 
the  task-based  constraints,  as  discussed  below,  are  used  to 
reduce  the  redundancy  of  the  problem. 

•  Initial  posture :  The  initial  posture  of  the  digital  human 
is  given  by  the  user.  Depending  on  the  task  requirement, 
all  or  some  of  the  joint  variables  can  be  assigned.  We  also 
assume  that  the  motion  starts  from  a  static  pose. 

•  Feet  positions  and  orientations :  The  global  coordinates 
and  the  angles  about  the  global  Z  axis  for  both  feet  are 
assigned  by  the  user.  The  foot  point  coordinates  are 
constrained  accordingly. 

•  Parabolic  projectile  equation :  For  the  object  to  hit  the 
target  point,  the  release  position,  release  velocity,  and 
flight  time  must  satisfy  the  projectile  equation. 

•  Hand  release  orientation :  At  the  release  point,  the 
direction  of  the  palm  should  be  the  same  as  the  direction 
of  the  release  velocity. 

•  Overhand  throw :  As  part  of  the  overhand  requirement, 
the  global  y  component  of  the  right  hand  velocity  should 
be  always  positive. 

•  Visual  perception :  This  constraint  requires  that  the 
target  point  should  exist  within  the  visual  field  of  the 
human.  The  hand  position  at  the  release  point  should  also 
be  located  within  reasonable  visual  range  so  that  the 
human  can  use  the  perception  to  control  the  release 
movement. 

Fig.  6  shows  sequential  snapshots  of  Santos  throwing  an 
object.  Visually,  the  motion  appears  close  to  the  baseball 
pitching  motion.  The  resulting  simulation  shows  that 
Santos  tries  to  generate  torques  with  the  help  of  the  full 
upper  body  to  create  the  force  necessary  for  throwing. 


Execution  starts  Release  point  Follow-through 


Fig.  6  Snapshots  of  Santos  throwing  an  object 


Hi.  Box  lifting  formulations 

Simulation  of  the  box-lifting  task  requires  the 
specification  of  initial  and  final  box  locations  as  well  as 
the  weight  of  the  box  to  be  lifted.  Several  task-based 
constraints  as  described  below  are  implemented  in  this 
work  to  satisfy  boundary  conditions  throughout  the  lifting 
process  (See  Fig.  7(a)): 

•  Hand  orientation :  Two  hands  are  normal  to  the  box  to 
facilitate  the  grasping  postures  for  hands. 

•  Vision :  The  vision  constraint  aligns  the  vision  vector 
towards  the  box  center. 

•  Collision  avoidance :  The  collision-avoidance  constraint 
is  used  to  keep  the  box  from  penetrating  the  body. 


Fig.  7  Box-lifting  task  (a)  Hand  orientation,  vision,  and 
collision-avoidance  constraints  (b)  Input  parameters 

The  lifting  task  is  to  lift  the  box  from  the  initial  location 
to  the  final  location.  The  initial  location  is  hi  high  from 
the  ground  and  di  away  from  the  feet;  the  final  location  is 
h2  high  from  ground  and  d2  away  from  the  feet,  as 
depicted  in  Fig.  7(b).  The  location  parameters  and  box 
weight  (w)  are  input  parameters  for  the  task. 
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Fig.  8  Sequential  snapshots  of  Santos  moving  a  10-lb  box 
from  a  lower  shelf  to  a  higher  shelf  (a)  without  any  torque 
limits  on  the  spine,  and  (b)  with  torque  limit  on  the  spine 
(time  progression  from  left  to  right) 

Fig.  8  shows  the  results  of  a  box-lifting  prediction  for 
lifting  a  10-lb  box  from  a  lower  shelf  to  a  higher  shelf. 
The  sequential  snapshots  in  Fig.  8(a)  show  that  much  of 
the  dynamic  effort  is  contributed  by  the  spine.  However, 
when  torque  limits  are  applied  to  the  whole  body,  the 
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resulting  simulation  in  Fig.  8(b)  shows  contributions  from 
knee  and  hip  torque  rather  than  spine  torque  to  lift  the  box. 

3.3  Integration  with  Other  Components  of  Human 
Modeling 

Motion  prediction  is  the  keystone  of  the  Santos  human 
model;  other  components  either  affect  motion  or  draw  on 
the  output  from  motion  prediction,  as  shown  in  Fig.  9. 
When  metabolic  energy  is  used  as  a  performance  measure 
with  predictive  dynamics,  then  various  physiological 
indices  (heart  rate,  oxygen  consumption,  body 
temperature,  etc.)  can  be  determined  (Mathai,  2005). 
Joint  torque,  joint  angles,  and  body  position  from 
predictive  dynamics  can  be  used  to  conduct  ergonomic 
analysis.  Actuation  joint  torques  from  predictive 
dynamics  can  be  used  to  determine  muscle  force  (Patrick, 
2005),  which  can  then  be  used  to  calculate  muscle  stress 
and  displacement  (Zhou  and  Lu,  2005). 

The  following  two  sections  will  describe  in  detail  how 
clothing  affects  motion,  and  how  strength  and  fatigue 
models  can  be  linked  to  motion.  Ultimately,  we  are  able 
to  indicate  not  just  when  Santos  can  or  cannot  complete  a 
task,  but  how  Santos  alters  his  performance  based  on 
strength  and  fatigue,  or  various  clothing  designs. 


Fig.  9  Multiple  components  of  human  modeling 

4.  Muscle  Strength  and  Fatigue 

Muscle  modeling  includes  muscle  strength  and  muscle 
fatigue.  In  this  section  we  present  the  advances  in  muscle 
modeling. 

4.1  Muscle/  Joint  Strength 

In  vivo  muscle  force  is  a  highly  nonlinear  phenomenon 
that  is  dependent  on  factors  such  as  muscle  length, 
contraction  velocity,  and  past  contractile  history  (e.g., 
fatigue).  While  muscle  force  has  been  modeled  as  a  linear 
system,  linear  representations  are  not  as  accurate  as  more 
complex  nonlinear  models  (Frey  Law  and  Shields,  2006). 
Force  decays  nonlinearly  with  increasing  velocity  (Hill, 
1938).  Active  muscle  force  varies  with  muscle  length,  due 
to  varying  overlap  of  the  force-producing  filaments  (i.e., 
actin  and  myosin),  and  stretching  of  structural  proteins  at 
long  muscle  lengths.  It  is  not  a  simple  transformation  to 
apply  these  principles  from  the  single  muscle  level  to  the 
joint  level  due  to  multiple  synergistic  muscles  acting  at  a 
joint  and  the  varying  muscle  moment  arms  with  joint 


angle.  Our  approach  to  modeling  joint  strength  inherently 
includes  each  of  these  nonlinearities. 

We  are  experimentally  measuring  joint  peak  torque  at 
several  angles  through  the  normal  range  of  motion  and  at 
several  angular  velocities  (e.g.,  0  -  300°/sec)  for  6  major 
body  joints.  These  data  are  used  to  create  3D  surfaces, 
with  peak  torque  (strength)  as  a  function  of  joint  position 
and  angular  velocity  (Laake,  2007).  While  previous 
authors  have  reported  similar  3D  representations  of  joint 
strength  (Anderson  et  al.,  2007),  no  one  has  developed  a 
normative  database  to  use  for  digital  human  modeling. 
This  database  will  allow  us  to  represent  human  strength 
capability  as  percentiles,  similar  to  how  anthropometry  is 
represented,  e.g.,  5th,  25th,  50th,  75th,  and  90th  %iles  of 
human  strength  capability  for  men  and  women.  Note  that 
these  population  %iles  are  unrelated  to  anthropometric 
(height)  percentiles,  as  tall  individuals  can  be  relatively 
weak  or,  conversely,  short  individuals  relatively  strong. 

a.  Incorporating  Strength  into  Santos 
Santos  predicts  the  required  joint  torques  (versus  time), 
along  with  joint  angle  and  velocity  needed  to  accomplish 
a  given  dynamic  task.  If  we  plot  the  predicted  joint  torque 
versus  joint  angle  and  angular  velocity  at  each  point  in 
time,  we  can  assess  the  magnitude  of  the  predicted  joint 
torque  relative  to  a  population  percentile’s  maximum 
capability  (i.e.,  50th  %ile  male  normative  3D  surface). 
This  provides  a  model  of  percent  effort,  where  the  closer 
the  predicted  task  lies  to  the  maximum  surface,  the  more 
difficult  a  task  becomes.  This  provides  a  unique 
methodology  for  a  digital  human  to  predict  the  perceived 
level  of  difficulty  of  a  subtask,  incorporating  known 
muscle  force  determinants  such  as  muscle  length,  moment 
arm,  and  shortening  velocity  into  one  simple  process.  To 
demonstrate  this  process,  walking  with  and  without  a  40- 
lb  backpack  was  modeled,  with  the  resulting  knee  joint 
position,  velocity,  and  torque  predictions  shown  in  Fig.  10 
a-c.  Fig.  lOd  plots  this  data  relative  to  the  5th-percentile 
male  strength  surface  (knee  flexion),  showing  that 
walking  with  a  40-lb  backpack  becomes  nearly  maximal 
intensity  for  the  weakest  5%  of  males.  Although  not 
shown,  this  task  is  only  moderately  challenging  for  the 
50th-%ile  strength  male  population,  and  easy  without  the 
backpack. 

4.3  Muscle/  Joint  Fatigue 

As  muscles  fatigue,  both  maximum  torque  and  velocity 
are  impacted,  e.g.,  it  becomes  increasingly  more  difficult 
to  generate  large  joint  torques  and  high  movement 
velocities.  Thus,  we  can  use  the  3D  joint  strength  surfaces 
to  represent  fatigue  by  decaying  them  with  repetitive  or 
higher  intensity  activities.  We  have  developed  a  model 
which  predicts  how  a  joint  surface  will  decay  over  time, 
using  a  series  of  differential  equations  based  on 
compartment  and  control  theories  (Xia  and  Frey  Law, 
2008).  A  single  three-compartment  model  represents 
muscles  involved  at  a  joint  in  one  of  3  states:  active, 
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resting,  or  fatigued.  Rate  constants  define  the  behavior  of 
the  transfer  between  the  active  and  fatigued  compartments; 
however,  we  use  a  proportional  controller  to  define  the 
transfer  between  resting  and  active  states.  The  model 
determines  how  much  of  the  resting  muscle  pool  must  be 
activated  in  order  to  match  the  predicted  joint  torques. 
The  combined  size  of  the  resting  and  active  pools 
determines  the  residual  capacity  of  the  system  for  use  as  a 
decay  coefficient  (values  between  0  and  1)  to  decay  the 
3D  strength  surface.  This  can  be  used  both  as  a  feedback, 
post-processing  mechanism,  providing  a  means  to 
measure  “time  to  fatigue”  when  the  task  is  no  longer 
feasible  without  alterations  in  the  predicted  dynamics; 
and/or  a  feed-forward  mechanism  where  the  predicted 
dynamics  can  change  as  the  available  strength  decreases. 
Most  notably,  we  incorporate  strength  nonlinearities  into 
or  fatigue  model  by  normalizing  predicted  dynamic  joint 
torques  by  the  corresponding  peak  3D  strength 
representation  (%  of  maximum  torque).  Thus,  the  model 
targets  this  %  max  torque  rather  than  a  specific  absolute 
muscle  force  or  torque,  as  typically  used  by  other  models. 
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Fig.  10  Predicted  gait  dynamics:  A)  knee  angle,  B) 
velocity,  and  C)  peak  torque  per  stride  length  with  and 
without  a  40-lb  backpack.  D)  The  40-lb  gait  data  vs.  the 
male  5th  %ile  strength  surface  for  knee  flexion. 


5.  Human- Clothing  Interaction  Modeling 


In  the  present  framework,  the  clothing  is  modeled  as 
flexible  continuum  shells  draped  onto  an  active  piecewise 
rigid  human  body  surface.  The  clothing  model  undergoes 
unilateral  frictional  contact  with  the  moving  human  body 
model  and  exerts  associated  forces  on  the  human  body. 
These  forces  can  then  be  used  to  calculate  the  energy 
necessary  to  move  the  clothing,  which  can  be  substantial 
with  heavy  protective  suites.  Here,  we  describe  how  a 
subject-specific  human  body  surface  driven  by  predictive 
dynamics  is  approximated  with  piecewise  rigid  mesh 
segments  and  how  the  clothing  is  draped  onto  the  body 
surface.  Although  not  the  focus  here,  constitutive 
behavior  of  different  clothing  fabrics  is  an  important 
aspect  of  clothing  modeling  (Swan  et  al.,  2007). 


The  starting  point  for  a  virtual  mannequin  representing 
the  anthropometry  of  a  specific  human  subject  is  a  laser 
body  scan,  which  yields  a  polygonal  mesh  (Ashdown, 
2007)  involving  hundreds  of  thousands  of  nodes  and 
polygons.  Such  meshes  are  much  finer  than  is  necessary 
for  clothing  modeling,  and  indeed  usage  of  such  fine 
meshes  would  be  too  computationally  expensive  in 
clothing  contact  modeling.  Accordingly,  the  body-surface 
mesh  can  be  coarsened  using  commercial  software  tools 
(Fig.  11).  Once  a  complete  and  coarsened  body  scan  mesh 
is  obtained,  it  is  decomposed  into  an  assemblage  of 
individual  meshes  corresponding  to  limb  or  torso 
segments.  The  optimal  decomposition  of  meshes  into 
segments  can  itself  be  quite  involved  (see,  e.g.,  Lien, 
2006)  but  was  done  here  (Fig.  lie)  in  an  ad-hoc  manner 
at  the  major  joint  locations  using  AutoCAD. 

The  individual  body  mesh  segments  are  subjected  to  rigid 
body  translations  and  rotations  in  time  to  approximate 
evolution  of  the  body  surface  as  the  human  subject 
performs  physical  tasks.  Geometric  inconsistencies  will 
develop  at  the  joints  between  the  rigid  body  mesh 
segments  (Fig.  12a),  and  since  such  gaps  present  a 
problem  in  clothing  modeling,  they  are  patched  in  the 
current  framework  using  auxiliary  spherical  and 
ellipsoidal  rigid  mesh  segments  at  each  of  the  joints 
between  body  mesh  segments  (Fig.  12b). 


(a)  (b)  (c) 

Fig.ll.  (a)  Original  body  scan  mesh  of  230,000 
polygons;  (b)  coarsened  mesh  of  only  1,700  polygons; 
and  (c)  coarsened  mesh  decomposed  into  segments. 

Fig.  12  (a)  Virtual  mannequin  in 
action  and  developing  gaps  at 
the  joints,  specifically  here  at 
the  left  shoulder  and  right  knee; 

(b)  Virtual  mannequin  with 
rigid  spherical  and  ellipsoidal 
joint  segments  that  become 
exposed  when  the  gaps  would 
otherwise  form  between  the 
rigid  body  mesh  segments. 

With  a  controllable  piecewise  rigid  body  surface  for  the 
mannequin  in  place,  clothing  models  can  be  draped  onto 
the  mannequin.  A  number  of  different  approaches  have 
been  taken  to  get  the  clothing  model  onto  the  human 
body.  Two  in  particular  are  trying  to  simulate  the  actual 
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dressing  process  (Man  and  Swan,  2007;  Volino  and 
Magnenat-Thalmann,  2000)  and  trying  to  pre-position 
models  of  clothing  patterns  around  models  of  the  human 
body  and  then  bring  the  patterns  together  and  stitch  them 
up  at  their  seams  (Gro(3  et  al.,  2003),  essentially 
constructing  the  garment  about  the  mannequin.  Here,  we 
try  a  different  approach  that  involves  two  steps:  (1)  the 
clothing  is  statically  pre-positioned  over  the  mannequin 
without  any  concern  for  contact  or  penetration  between 
the  body  and  the  clothing;  and  (2)  once  the  clothing 
model  is  in  place,  contact  mechanics  (penetration 
detection  and  correction)  are  turned  on  to  eliminate 
clothing  penetrations  of  the  body  surface. 


When  pre-positioning  the  assembled  clothing  models 
about  the  body  in  a  fixed  posture,  the  objective  is  to  bring 
the  centroids  of  the  clothing  model  edges  (cuffs, 
waistline,  necklines,  etc.)  into  alignment  with  the 
corresponding  body-segment  edge  centroids.  This  is 
achieved  by  using  penalty  forces  that  are  significant  when 
the  clothing  and  body  segment  edge  centroids  are  not 
coincident  and  are  minimized  as  the  clothing  becomes 
properly  positioned  on  the  body.  This  pre-positioning 
problem  is  solved  quasi-statically,  leaving  the  clothing 
model  properly  positioned  on  the  body  in  a  gross  sense, 
although  with  some  significant  clothing  penetrations  of 
the  body  (Fig.  13). 


Fig.  13  Results  of  the  clothing  pre-position  step  where, 
without  any  concern  for  penetrations,  the  edge  centroids 
of  the  garment  models  are  brought  into  alignment  with  the 
edge  segment  centroids  of  the  corresponding  body 
segments;  clothing  penetration  of  the  body  is  especially 
evident  at  the  shoulders  and  tops  of  arms 


In  the  second  stage,  explicit  dynamic  analysis  of  the 
clothing  model  is  performed  while  gravity  loading  pulls 
downward  on  the  clothing  model,  and  explicit  contact 
analysis  as  described  by  Man  and  Swan  (2007)  is  utilized 
to  remove  starting  penetrations  and  any  other  penetrations 
that  develop.  Virtually  all  of  the  initial  clothing 
penetrations  from  the  pre-positioning  stage  are  eliminated 
in  this  phase  of  analysis  (Fig.  14).  From  this  stage,  the 
mannequin  can  be  activated  based  on  joint- angle  profiles 
from  predictive  dynamics,  and  the  clothing  model  will 
respond  accordingly  via  frictional  contact  interactions 
with  the  mannequin  surface. 


Fig.  14  Result  of  second  stage 
analysis  in  which  gravity 
loading  is  applied  along  with 
contact  mechanics  to  remove 
initial  clothing  penetrations  of 
the  body 


Conclusions 

This  paper  presents  new  developments  for  the  digital 
human  Santos.  These  new  capabilities  include  a  high- 
fidelity  skinned  kinematic  skeleton  model  with  over  two 
hundred  DOFs,  advances  in  muscle  strength  and  fatigue 
modeling  (working  at  the  joint-space,  incorporating  3D 
dynamic  joint  torque  and  the  development  of  a  new 
fatigue  model),  advances  in  cloth  modeling  (human  and 
cloth  interaction  modeling),  and  advances  in  predictive 
dynamics  (simulation  for  different  tasks  such  as  walking 
on  stairs,  throwing,  and  box  lifting).  In  addition  to  these 
new  developments,  a  variety  of  inter-related  research 
efforts  are  ongoing:  hand  modeling,  such  as  grasping  and 
the  development  of  hand  comfort  metrics;  advanced 
posture  prediction  (whole-body  posture  with  global 
translation  and  rotation  DOFs)  for  the  new  skeletal  model; 
development  of  a  zone  differentiation  tool;  predictive 
dynamics  tasks  such  as  kneeling,  side  walking,  ladder 
climbing,  and  ingress/egress;  further  refinement  of  the 
polygonal  topology  to  allow  for  facial  expressions  and 
blinking  and  refinement  of  the  skin  weighting  to  address 
any  skin  tearing  issues  we  discover  as  we  begin  using  the 
avatars  with  the  new  kinematic  skeleton,  and  interface 
development. 

This  paper  illustrates  the  necessity  for  multi-disciplinary 
work  when  modeling  humans.  In  addition,  as  shown  in 
Fig.  10,  various  research-and-development  efforts  must 
all  link  together,  providing  a  comprehensive  human 
model.  The  joint-based  focus  of  our  model  helps 
facilitate  this  connectivity  between  various  aspects  of  a 
virtual  human. 
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